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(54) ANIMATION IMAGE RETRIEVING METHOD 

(57)Abstract: 

PURPOSE: To retrieve an optional scene by means of an index which can be 
understood by humans in a frame word. 

CONSTITUTION: The physical value showing the animation contents is sorted into 
plural categoriese.g. the size and the color of an objectetc. A word data base 
including plural words like 1arge"medium M smairred"yellow"green'etc.is prepared for 
expression of the preceding categories. Then plural works are automatically 
selected by a computer out of the word data base based on the physical value 
obtained through the processing of an animation. These selected words are applied 
as the indexes for each word frame. In a retrieving execution statethe word 
corresponding to the desired scene to be retrieved is inputted. Then the frame of 
the coincident animation image is defined as a candidate image. 



CLAIMS 



[Claim(s)] 

[Claim 1]Classify physical quantity expressing the contents of video into two or 
more categoriesand a word data base containing two or more words expressing 
these categories is preparedA computer is made to choose two or more words 
automatically out of said word data base based on physical quantity produced by 
processing videoA video search method showing a frame of video which makes a 
word which is equivalent to a scene to search from said word data base to a user 
at the time of retrieval execution chooseand in which two or more words made to 
choose are given per frameand the same word as the word made to choose is 
given as a candidate image. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application]This invention relates to the video search method used 
when searching and pulling out the head of the scene needed from a dynamic 
image data file. 
[0002] 

[Description of the Prior Art]The conventional video search method is search of 
the file basis which discovers the video file in which the required scene is likely to 
be contained using a keyword in seeing the representation screen manually given 
to the video file basis Thuswhen searching a certain scene in a file after 

discovering a fileit is carried out by the method of seeing the contents of the file 
from the head of a fileand finding out a required scene. Like drawing 8t he temporal 
change figure 2 of statistics valuessuch as luminance distributionis given to the 
representation screen 1 of a video file togetherand there is a method of seeing the 
statistics value and looking for a required scene out of a file. 
[0003] 

[Problem(s) to be Solved by the Invention]In such a Prior artsearch is possible 
only in the multiple frame unit of a video file. In order to enable it to refer to a fine 
unit using the representation screen 1 mentioned abovemany representation 
screens 1 will be shown at onceand the method of making a user choose from the 
inside will be taken. Howeversince the user has to look at a lot of representation 
screens 1 in order to acquire the target picture when referring to this methoda lot 
of time is needed. Since a keyword must be given per frame when using the 
keyword mentioned abovemany time and helps are needed for giving. 
[0004]The purpose of this invention is enabling it to search arbitrary scenes by 
providing the method of giving the index which can be understood to human being 
per frameand the search method using the index. 
[0005] 

[Means for Solving the Problem]Physical quantity which expresses the contents of 
video with a video search method concerning this invention as pretreatment in 
order to solve said SUBJECT is beforehand classified into two or more 
categoriestwo or more fundamental words for expressing the category are 
definedand it prepares as a word data base. And it is made to choose 
automatically about each category from word data bases mentioned above based 
on physical quantity generated by processing video by a single frame and order 
inter-frame. After doing this workthose words are given to video per frame. This 
method enables it to give an index automatically per frame. And it searches per 
frame by making a frame to which the same word as a word which made choose a 
word showing a scene to search for every category from word data basesand was 
chosen is given show as a candidate at the time of search. 
[0006] 



[Function]The word showing a scene to search with this invention for every 
category at the time of search may only be inputted directly or indirectly. 
Thereforesearch time is shortened also when it is going to search the arbitrary 
places in a video filewhile becoming possible to reduce the work done by human 
being. 

[0007] 

[Example]Belowone example of the video search method of this invention is shown. 
The video search method of this invention is roughly dividedand consists of twothe 
process in which choose two or more words used as the index for search from a 
word data baseand they are givenand the process in which it searches using the 
word. 

[0008]Firstan example of the generation method of a word is shown. Herethe 
following are prepared as an example of a word data base. The word of drawing 7 
is given to each category by making "the size of a photographic subject"the color 
of a photographic subjectthe movement speed of a photographic subjectand the 
"move direction of a photographic subject" into a category. Namelyin the example 
of drawing 7t hree words of small" are prepared for the category of size" of 
"photographic subject into "sizeHereafterto "the color of a copy body"the right, 
the upper right, a top, the upper left, the left, the lower left, the bottom, and the 
lower rightare further prepared [ "red yellowgreenblueand purple" ] for "a quick 
medium speed and late stillness" to "movement speed of a photographic subject" 
to "movement speed of a photographic subject" in a similar manner. 
[0009]Nexthow to assign the word by a computer is described. Since the above- 
mentioned category is related with a photographic subjecta photographic subject 
field is extracted for every frame. The extraction methods of a photographic 
subject field are methodssuch as "1 examination (the 1991 Akinobu **** sizeD- 
200-1991) about moving object field extraction"for exampleand can be carried out 
by specifying a photographic subject field about a head frame. After performing 
field extraction by these methodscharacteristic quantitysuch as barycentric 
coordinates of the extracted fieldareawidthheightand hueis calculated from each 
field. And a word is assigned based on the physical characteristic quantity which 
was carried out in this way and calculated. An example of the method of assigning 
the word about the movement speed and the move direction of a photographic 
subject using the value of a centroid position is explained using drawing 1 . the 
start — movement — a direction — a deciding method — ****** — first — 
drawing 1 — it can set — a rear frame — 12 — a previous frame — 11 — the 
center of gravity — P — __ 12 __ — P — __ n _ — being shown — coordinates — x - 
one _ ~ x — _ , _ . — y — s __ one — y — _ i — from — the following 
— ( — one — ). (2) Using a formulaask for dXj and dyjrespectively and the numerals 
of dxj and dyj determine the word of the move direction. 
[0010] 
[Equation 1] 



In the example of drawing 1 if the lower left is made into an origin of 
coordinatessince it will be set to dXj>0 and dy^Othe word the "upper right" can be 
assigned out of the category of the word of the "move direction of a photographic 
subject" of drawing 7 . Movement speed asks for drj using (3) typesand assigns the 
word about "movement speed of a photographic subject "for examplemedium 
speedfrom the value of the dr^ 
[0011] 
[Equation 2] 

Hereafterthe word about a colorfor exampleredand the word about area to a 
sizefor exampleinsideare similarly assigned from hue data. 

[0012]Nextan example of the search method using these words is described. The 
flow chart of the retrieving procedure described below is shown in drawing 2 . (1) - 
(10) shows each step among drawing 2 . A retrieving procedure begins from 
specifying first the word which is an index of the video which a user wants to 
search. The appointed method has directly the method (1) of specifying a 
word(2)and the method (1) of pointing to the sample scene which is a similar 
sceneand specifying a word indirectly(9)(10) and (2). After specifying a wordsearch 
of a step (2)(3)and (4) is started. (8) and search will be ended if the scene which 
will point to (5)(6)and its scene to a user as a candidate's sceneand a retrieving 
person will mean (7) and in it if the word given to the video in a database and the 
word specified by a user are in agreement is included. 

[0013]An example of the user interface for specifying a word is shown in drawing 3 . 
As for a retrieval execution button and 23in drawing 3 21 is [ a search-results 
display window and 25 ] search-results check windows a sample scene display 
window and 24 the word sequence appointed window and 22. How to specify a 
word directly using the interface of drawing 3 is described below. 
[0014] Drawing 4 is an enlarged drawing of the word sequence appointed window 21 
in drawing 3 . In drawing 4 21 1 shows pointing cursor and 212 shows the button 
display for word specification. When specifying a word directlyonto the icon of the 
word which is expressing the scene in the word sequence appointed window 21 to 
search like drawing 4 a mouse etc. are usedcursor is movedand a word is specified 
by clicking on an icon. The word specification method using a similar scene is 
performed using two user interfacesthe sample scene display window 23 of 
drawing 3 and the word sequence appointed window 21 mentioned above. 
[0015] Drawing 5 is an enlarged drawing of the sample scene display window 23 in 
drawing 3 . 

The file name appointed window and 233 show a sample scene234 shows a scroll 
barandas for a control panel and 232231 shows a control button 235. 

[0016]Firsta file name is specified in the file name appointed window 232 in the 
sample scene display window 23 of drawing 5 If the sample scene 233 is displayed 



with the reproduction button and fast forward button in the control panel 231 and 
the scroll bar 234the word corresponding to the scene will be displayed on the 
word sequence appointed window 21 of drawing 4 . When the contents displayed on 
this word sequence appointed window 21 differ from the contents which a 
retrieving person meansit is possible to correct by the method same like drawing 4 
as the case where a word is specified directly. 

[001 7]As stated aboveafter specifying a wordon the retrieval execution button 22 
in drawing 3 a mouse etc. are usedcursor is movedand search is started by clicking 
a button on it. A search method compares the word currently displayed on the 
word sequence appointed window 21 with the word beforehand given by the 
method mentioned above in the picture in an image databaseand displays the 
scene whose word corresponded on the search-results display window 24 like 
drawing 3 . When you would like to see in more detail the scene displayed on the 
search-results display window 24 of drawing 3 Cursor is moved with a mouse etc. 
onto the scene in the search-results display window 24 to seeand the scene 
clicked by clicking the button of a mouse is displayed on the search-results check 
window 25 of drawing 3 . The enlarged drawing of the search-results check window 
25 is shown in drawing 6 . A reproduction buttona fast forward buttonetc. in a 
figure can perform the check of a result. When it differs from the scene which the 
scene displayed on the search-results display window 24 meantthe target scene 
can be searched by specifying a word again in the word sequence appointed 
window 21 like drawing 4 and reperforming a search. 
[0018] 

[Effect of the Invention]As explained to details abovethe physical quantity 
expressing the contents of video is classified into two or more categories 
according to this inventionThe word data base containing two or more words 
expressing these categories is preparedA computer is made to choose two or 
more words automatically out of a word data base based on the physical quantity 
produced by processing videoGive two or more words made to choose per 
framethe word which is equivalent to a scene to search from a word data base to 
a user at the time of retrieval execution is made to chooseand the frame of the 
video in which the same word as the word made to choose is given was shown as 
a candidate image. 

Thereforesince an index can be given in arbitrary units (for exampleframe unit) 
while being able to make the great portion of indexing work manually done to the 
former do on a computerit becomes possible to search the arbitrary scenes in a 
video file. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] It is an explanatory view of the word generation method in this 



invention. 

[Drawing 2] It is a figure showing the example of the flow chart of the image 
retrieval in this invention. 

[Drawing 3] It is a figure showing the example of the interface screen for the image 
retrieval in this invention. 

[Drawing 4] It is a figure showing the example of the word sequence appointed 
window in this invention. 

[Drawing 5] It is a figure showing the example of the sample scene display window 
in this invention. 

[Drawing 6] It is a figure showing the example of the search-results check window 
in this invention. 

[Drawing 7] It is a figure showing an example of the database which consists of a 

category of the physical quantity expressing the contents of the photographic 

subject in this inventionand a word expressing these categories. 

[Drawing 8] It is a figure showing the example of the cine mode display using the 

conventional statistics value. 

[Description of Notations] 

1 1 Previous frame 

1 2 Rear frame 

21 Word sequence appointed window 

22 Retrieval execution button 

23 Sample scene display window 

24 Search-results display window 

25 Search-results check window 



